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A teacher's manual for operating the DX System, a 
software package for use in diagnostic testify programs, is 
presented. The DX System is constructed to present test items based 
on the individual student's pattern of responses. Described are: (1) 
system components designed to run on an Apple II computer; (2) 
program options allowing the user to administer tests, collect 
summary reports, and input new and revised tests; and (3) a system 
demonstration taking the user through a step-by-step test 
administration and summary of results, using two prototype tests 
measuring student performance in pronoun usage and reading 
comprehension. Included in a section on DX System implementation are 
subdivisions on test construction, entry of test items, test 
administration, summaries of results, and teaching and retesting. 
Samples are given showing test item layout and test performance 
summaries. A flow chart of test item presentation follows the text. 
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INTRODUCTION TO THE DX SYSTEM 

T H - DX System is a software package for use in diagnostic testing 
programs. It is designed for individual student assessment of specific 
skills and/or content within identified domains. Student performance 
within a domain is diagnosed by assessing both requisite and prerequisite 
skills during a specially-designed presentation of test items at various 
levels of difficulty. 

The DX System is constructed to present test items based on the 
individual student's pattern of responses. This capability of 
individualized testing allows the DX System to identify each student's 
strengths and weaknesses within a domain. In addition to skill need 
identification, the DX System is capable of recording and integrating test 
results to provide a summary report of student performance. These reports 
are essential for planning a complete diagnostic-prescriptive program of 
instruction. 

Trie DX System gives the user several kinds of diagnostic information. 
The key features of the summary data for each student include 
identification of performance level, specific skill needs, and types of 
errors made. In order for this information to be accurate, tests must be 
constructed using guidelines and directions provided in the diagnostic test 
construction manual. 

In summary, the DX System is a specially-designed adaptive diagnostic 
testing package that combines educational diagnosis, psychometric theory, 
and computer technology. As a diagnostic tool, the DX System identifies a 
student's strengths and weaknesses and provides specific information for 
subsequent prescription. The flow of test item presentation reflects key 
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psychometric considerations— -multiple levels of difficulty, distractors 
related to error types, and decision paths. The interaction and 
information-processing needed for sophisticated diagnostic testing are best 
accomplished in a computerized approach (McArthur * Choppin, 1984). 1 Thesr 
features are efficiently handled by the DX Systt- through the use of Pascal 
programming and the Apple computer. 

The DX System has been developed for use in programs requiring a 
systematic, comprehensive approach to diagnosis. Tests constructed for use 
in the DX System must follow specified guidelines. Two prototype tests are 
currently in the system for use in demonstrations and training sessions. 
These tests measure student performance in pronoun usage and reading 
comprehension. 



McArthur, D. L., & Choppin, B. H. (1984). Computerized diagnostic 

testing. Journal of Educational Measurement , Vol 21(4), 391-397. 
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DX SYSTEM COMPONENTS 

The DX System contains three diskettes: 

Diskette A — the system files 
Diskette B — the test files 
Diskette C— the test ed^t files 

The DX system runs on an Apple II (II plus, He) computer. System 
requirements include a memory card to increase memory to 64K, a video 
monitor, and two disk drives. A printer is optional. 

The three diskettes included in the package provide all the necessary 

program components. As multiple tests are constructed, however, additional 

test files diskettes (Diskette B) will need to be made. Setting up test 
files is addressed in Section V. 
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DX PROGRAM OPTIONS 

The DX System allows the user to administer tests, collect summary 
reports, and input new and revised tests* The names for these options are 
described below. 

1) RUNTEST- 

RUNTEST is the name of the file that is used for 
test administration. RUNTEST administers tests, 
accepts students' responses, delivers feedback and 
tracks student performance to make decisions on the 
testing sequence. RUNTEST is also the word that is 
entered at the keyboard to start the testing 
process. This option is demonstrated in Section IV. 

2) SUMMARY- 

SUMMARY is the name of the program that processes 
students' records of test performance. SUMMARY 
analyzes students' responses and error patterns and 
presents individual performance summaries. SUMMARY 
is also the word that is entered at the keyboard in 
order to see a dispUy of test results. See Section 
IV for a demonstration of SUMMARY. 
**Diskettes A and B are used when running RUNTEST and SUMMARY.** 

3) EDITTEST- 

The EDITTEST file on Diskette C allows the user to 
input new and revised test items. This process is 
explained in Section V. 
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DX SYSTEM DEMONSTRATION 



This section takes you step-by-step through test administration and 
summary of results, using the pronoun and comprehension tests provided on 
Diskette B. 

A. BEFORE YOU BEGIN - An important preliminary step is making back-up 

copies. Follow the directions in your Apple user's manual 
to copy diskettes A, B, and C. Store the back-up diskettes 
for use in the event that an original diskette fails. 

B. TEST ADMINISTRATION - 

Preparation 1 - With the computer off, insert diskette A (label side u£) 

into drive #1 and cFTskette B (label side ip) into drive 
#2. 

2 - Next, turn on the computer and the video monitor. 

3 - In a few moments, the screen displays the following 

command selections: 

Command: E(di t R(un Fu'le C(omp L^'nk 
X(ecule A(ssem D(ebug? 

Type X for execute. Your command is immediately 
processed. 



4 - When asked "Execute what file?" type RUNTEST and press 
RETURN. 
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Test-taking 1 - The test session begins with the message: 

"Please type your first and last name and finish with 
the RETURN key" 

The name entered at this point becomes the filename for 
the student's file of test results. Students should 
enter first and last names. 



2 - The next screen display greets the student and 

introduces the test. The information is displayed for a 
short time and then the first pronoun cest item appears. 



3 - For each test item, select the letter of the correct 
response. As soon as a letter is typed the program 
processes the response and provides immediate feedback. 
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4 - If a student types a letter other than the available 

choices, this message appears: 

"Try again . • . Choose one letter for the best answer." 

After three entry errors, the correct response 1s 
displayed ana the next test item is presented, 

5 - Depending on the pattern of responses, up to 12 test 

Items are presented. Arter the final item, results for 
the pronoun test are recorded and stored on diskette B. 
As this process is being done, you see the message: 

'Please wait" 

The comprehension test follows Immediately after this 
sUp. 

6 - Continue answering comprehension questions as before. 

Depenc^ng on the pattern or responses, up to 12 items 
will he presented. (Lee appendix A for a description of 
tne f?ow of test item presentation.) 

After the last comprehension test item the "Please 
wait" message is displayed while test results are 
recorded. 

7 - The testing session is finished when this message 

appears: 

"Thank you" 



11 



8 



C. SUMMARY OF TEST RESULTS 

Viewing the 1 - To execute the SUMMARY program, type the X command when 
Results this line is displayed - 

Command: E{dit S'un F(i1e C(omp L(ink 
X(ecute A(ssem D(ebug? 

2 - When aske< "Execute what file?" type SUMMARY and press 
RETURN 

3- The Catalog of test results files is then displayed on 
the screen. Note that entries ~~e written in the 
following form - 

first initial, last name, fir^t letter of tes**„ 

Find your filename and type it exactly as it is listed 
on the catalog. An example is displayed on the screen. 
When finished, press RETURN. 

4 - At the bottom of the screen is the question: 

"Do you yant to print this student's summary? ( Y/N ) 11 
If no, typt, N and press RETURN. If yes, TURN 1 PRINTER 
ON and then type Y and press RETURN. 

5 - ^he results of the test you selected a^e displayed on 

the screen. Figure 1 contains some additional sample 
test summar ies. 
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Note the following types of data: 

- # of items attempted & # of items correct 

- % correct overall 

- % correct within levels 

- number and type of errors made 

- the test items for which the errors apply 

6 - Wnen finished printing or review"ng results, press 
RETURN to return to catalog or ' ype the FILENAME you 
wish to review next. If completely fi.ished type BYE 
and p^ess RETURN. 

Summary information is stored on Diskette B until the 
file is deleted. Up to 12 student files (depending on 
the length of the test? on the diskette) can be stored 
on a diskette. To delete a student file, follow this 
sequence: 

1. When the Command selection is displayeG, type £ for 
File 

2. Next, type R when the list of files commands Is 
di splayed. 

3- When "Remove?" is displayed, type #5: FILENAME. TEXT 
(use the filename to be deleted). 

4- The question "Update directory?" allows you to 
check your entry before removal. If th^s is the 
cor r ect file to be deleted, type Y. If not 
type N and repeat step 2 through 4. 

5- Type Q for quit. Wnen the command line appears 
either execute (X) a new file or turn off the 
compute . 

In actual use, the procedures for *est administration, summary of 
results and removal of student files, remain constant regardless of the 
test being used. 
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DX SYSTEM IMPLEMENTATION 

The complete DX System Implementation includes: 

A. Test construction 

B. Entry of test items 

C. Test administration 

D. Summary of results 

E. Teaching and re-testing 

A. Guidelines for diagnostic test construction are provided in the manual 
entitled "Test Design Manual: Guidelines for Developing Diagnostic 
Tests". The DX program is designed to Vit a wide range of testing 
protocols, excluding only hose tests which require free response. When 
constructing diagnostic tests for the DX system, certain considerations 
must be followed. These are described in Part B. 

B. The design the DX System requires a diagnostic test with the 
<V cing features (see * 2): 

- 3 levels of u. nculty (or skill type) 

- 2 passages (sentences, problems) per level 

- 3 cr-stions per passage - each identified by error type 
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This totals 18 questions for each test. Each complete test item (1 
passage, 3 questions with answer choices) follows the layout presented 
in figure 3. There is a maximum of 20 lines available on the screen 
display for a passage and a question. Therefore, text portions should 
be limited to 15 lines and each question and answer segment should not 
exceed 5 lines. 

Note the "flags" that are included in the test item (e.g., *, *3303,)). 
These are essential for correct processing of the test items by the DX 
System. 

To view, input and revise test items, follow these steps - 

1. Insert diskettes A and B as before and turn on computer. 

2. Type X for execute. 

3. Remove diskette B from drive #2 and insert diskette £. 

4. When asked, "Execute what file?" type #5: EDITTEST 
and press RETURN. 

5. A menu of 4 choices is displayed. Insert the letter of 
your choice and press RETURN. 
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Figure 1 



Sample text showing test Item layout 

Reading Ocr^> ebension Test 



There are many #~iive volcanoes on earth such as t 
Loa in Hawaii* They occasionally shoot smoke and lava ont 
surface. The formation of a volcano can be thought of as 
balloon buried under thin layers of sand and plaster. Hot 
balloon, and pushes up the ground. Magma is the hot mater 
earth's center core. The iric gma pushes upward inside the v 
the surrounding rock and dirt. The melte ^ock and dirt i 
Eventually, the lava will push through the top of the moun 
action causes a volcanic explosion and forms a crater. Af 
or eruption, the lava cools off and the mountain sides shr 
is similar to a ballnsn shrinking after it pops* A volcan 
inactive when i t no longer produces eruptions. 
♦ «* 



he famous Mauna 
o the earth' s 
an underground 

magma fills the 
lal that forms the 
olcano and melts 
s called lava, 
tain. This 
ter the explosion, 
ink somewhat. This 
o is said to be 



a. 
b. 
c • 
d. 

#3303 



b. 
c. 
d. 

♦2C22 

a. 
b. 
c. 
d. 

♦0111 
) 



This passage is mostly about how.., 
volcanos are filled with lava, 
mountains are sometimes volcanoes, 
volcanoes are created, 
magma creates mountains. 

— ■ 

An active volcano. •• 
erupts every other mo th. 
erupts occasionally, 
never erupts, 
contains cold lava. 



Flag: end of Item text 



Flag: end of Question text 
_y plus response markers 
(i.e. Error types: 

"a" » 3 

•W« - 3 

»c n - correct response 
•tf" « 3 



According to the passage, when a balloon pops, its sides, 
shrink. 

become larger, 
crack. 

become rounder. 



^Butterflies are created through an int 
butterflies lay their eggs on leaves or tree b 
caterpillars. The caterpillars crawl Around t 
leaves. Eventually, the catet pillars w**ve th 
cocoon. The caterpillars sleep inside the coc 
cocoon, the caterpillar grows its butterfly wi 
awakens, the butterfly break* out of its cocoo 
eggs, the process will begin again. 
# 

The passage is mostly about now... 



Flag: end of last item 

eresting series of stages. Adult 
ranches. These eggs turn into 
he trees and bushes, feeding on 
emselves a silken shell called a 
oon for a f i v weeks. Inside the 
ngs, antennae and body. When it 
n and flies away. When it lays 
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higher difficulty 
moderate difficulty 
lower difficulty 



There are very active vole 
surface. The formation of 
buried under layers of san 



This passage is mostly 

a. volcanoes are fille 

b. mountains are soinet 

c. volcanoes are creat 

d. magma creates mount 



r 3303 
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6. If Writing new test items you will be asked to name the test 
file and use either an Existing file or entry from the 
Keyboard. Then you will enter prepared test items, following the 
sample format provided above. 

7. If making Chcnges or Reviewing existing tests, yo>j will 
need to identify the test file to be used. Type the test 

file to be used. Type the test name as it appears on the catalog 
and the test will be displayed. 

8. When rjltiple tests are to be used, make multiple copies of 
diskette B. It 1s recommended that each test go on a different 
copy of diskette B, so sufficient storage room is available for 
test results files. Be sure to label diskette carefully. 

C. Procedures for test administration are demonstrated in section IV. 
Prior to following those procedures, consider these preparatory steps 

1 - make additional test diskettes, as the storage capacity for 

test results files is limited; 

2 - have complete directions and sample items for students; 

3 - plan a routine and a schedule for the testing session, 

allowing sufficient time for each student to take the test 
in one sitting; 
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Figure 3 



Sanple test performance smraaries 



Performance summary for D.MACA: pronoun test 

Out of 11 items attempted . 3 items were correct 
27 7. correct over a 1 1 
33 7. correct at highest level attempted (2) 
20 7. correct at lowest level attempted (l) 

This student made 

3 errors in using nominative pronoun. 

An example of this is Item # 17 

2 errors in using direct object pronoun. 

An example of this is Item # 11 

3 errors in using pronoun as object of preposition. 

An example of this is Item # 16 



Performance summary for D.MACA: comprehension test 

Out of 6 items attempted , 5 items were correct 
B3 7. correct overall 
0 7. correct at highest level attempted (3) 
10'"> 7. correct at lowest level attempted (2) 

This student made 

A single error in literal comprehension of a sentence. 

It occurred in Item # i 



Performance summary for M.BACCUS: pronoun test 



Out of 12 items attempted , 9 items were correct 
75 7. correct overal 1 
66 7. correct at highest level attempted (2) 
83 7. correct at lowest level attempted (l) 

This student made 

A single error in using nominative pronoun. 

It occurred in Item # 10 
A single error in using direct object pronoun. 

It occurred in Item # 7 
A single error in using pronoun as object of preposition. 

It occurred in Item # 16 



Performance summary for M.BACCUS: comprehension test 



Out of 7 items attempted , 5 items were correct 
71 7. correct overall 
100 7. correct at highest level attempted (3) 
66 7. correct at lowest level attempted (2) 

This student made 

A single error in identifying the main idea. 
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D. Summary of test results can be retrieved any time after the testing 
session. When summaries are no longer needed, file c can be deleted to 
allow room for new test results files. However, if re-testing is 
desired, up to four files can be stored using the name filename (i.e., 
for the same student), 

E. Tests in the DX System can be used for diagnosis prior to instruction 
and also for post- testing, desired. Errors identified by the system 
provide information for ins jctional planning and determination of 
student master 
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The preceding figure shows diagrammati cally the flow of test materials 
administered to several hypothencal students. The first student, whose 
path is shown as a bold line, gives a correct response to every question. 
While diagnosti cally uninteresting, from the point of view of the DX system 
her route through the various test items is responsive to her performance, 
as follows: 

- Entry is made to a passage of middle difficulty, with its first 
question [ # III 1 ]. 

- Regardless of response, the sequence continues through three 
additional items, [ # III 2, III 3, and IV 1]. 

- Because she has totalled four correct wichln this level at this 
juncture, she is next p> ^ented a passage from a high level of difficulty 
[ill]. 

- Regardless of response, the sequence continues through three 
additional items, [ # I 2, I 3, and II 1]. 

- Because she has totalled four correct witl.in this level at t*is 
juncture (and there are no higher levels to administer) she exits from the 
test. 

The second case, shown by a solid line is of a student who fails every 
question presented. Again diagnostically uninteresting, nonetheless his 
passsge through the test is governed by his performance. His opening 
sequence Is identical for four items, until: 

- Because he has four wrong responses after # IV 1, he is next presented 
with a passage from a lower level of difficulty [ # V 1]. 

- Regardless of response, the sequence continues through three 
additional items, [ # V 2, V 3, and VI 1]. 
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- Because he has totalled four wrong within this level at this juncture 
(and there are no lower levels to administer) he exits from the test. 

The dotted lines show additional moves made by students with less 
clear-cut patterns of response. That is, some but not all answers are 
correct. Now diagnostical ly interesting, the patterns suv.h students take 
through the system arc evaluatable within the context of both the level of 
the item at which their errors occurred, and the type of error which was 
made. This latter component is found in the error coding attached to each 
question, of which che system keeps track during the test to tabulate 
relative frequencies of error types. 
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Flow of Test Item Presentation 
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